A multi-stage anticipated surprise model with dynamic expectation for economic decision-making

There are many modeling works that aim to explain people’s behaviors that violate classical economic theories. However, these models often do not take into full account the multi-stage nature of real-life problems and people’s tendency in solving complicated problems sequentially. In this work, we propose a descriptive decision-making model for multi-stage problems with perceived post-decision information. In the model, decisions are chosen based on an entity which we call the ‘anticipated surprise’. The reference point is determined by the expected value of the possible outcomes, which we assume to be dynamically changing during the mental simulation of a sequence of events. We illustrate how our formalism can help us understand prominent economic paradoxes and gambling behaviors that involve multi-stage or sequential planning. We also discuss how neuroscience findings, like prediction error signals and introspective neuronal replay, as well as psychological theories like affective forecasting, are related to the features in our model. This provides hints for future experiments to investigate the role of these entities in decision-making.


Online Appendices and Supplementary Information
Appendix 1: Reproducing the patterns of prospect theory in the AS model with surprise functions of general form Here we are showing that the predictions from the AS model conform to the empirical observation in Kahneman and Tversky (1979) for the type of problems described in Table 1.For the gain domain, it is useful to study how the surprise function varies with respect to  in the setting described in Table A1.

Reward
Probability Outcome 1 1/  Outcome 2 0 1 −  Table A1: The setting for studying the property of the surprise function in the gain domain.
Note that we set ̅ = 1 to simplify analysis.
In this setting, the surprise value ∆, as a function , is given by: ∆() =  ( The 2 nd and 4 th conditions mean that ∆() is positive and decreasing at small .The 1 st , 3 rd and 4 th conditions necessitate that ∆() reaches negative value at some intermediate values of , and then increases and reaches the value of 0 at  = 1.This results in a U-shaped ∆() that crosses zero at intermediate .See, for example, Figure 1.The value of  where ∆() = 0, aside from the trivial solution  = 1, corresponds to the probability where the subject switch from the gambling option to the certain option.For the special case where  = 1, it can be easily shown from (A1) that ∆( = 0.5) = 0.For the realistic case where  > 1, note that  is decreasing with , which means that ∆( = 0.5) < 0 and that ∆ crosses 0 at a smaller value of .
In the loss domain, the relevant setting is shown in Table A2.
The setting for studying the property of the surprise function in the loss domain.
Appendix 2: Reproducing the results for problem 10 in Kahneman's 1979 paper using the AS model Here we are showing that our sequential branching mechanism implies that for problem 10 in Kahneman's 1979 paper, consistent to the experiment observations and what PT proposed, it is effectively equivalent to the case when the 1 st stage of the problem is ignored.The branching scheme of a general version of problem 10 is shown in Figure S1.
Figure S1: The branching scheme for a general version of problem 10 in Kahneman and Tversky (1979).
The surprise value ∆   , as a function  2 , is given by: Note that the term (1 −  1 ) − (1 −  1 )( 1 ) is independent of  2 .This is intuitive because events involving the same branch and same expected value at the intermediate states should generate the same anticipated surprise.
Consider two options: one with  2 = , the other with  2 = .The difference in the surprise value    = ∆   () − ∆   () is given by: In Kahneman's 1979 paper, it was suggested that the problem is equivalent to the case when the common blue branch is removed.When the common branch is removed, the problem reverts to the one that we discussed in Section 1.The surprise ∆   ( 2 ) is given by eq.(A1) in Appendix 1.The difference in the surprise value for the same two options    = ∆   () − ∆   () is given by: The option preference is determined by the sign of .Here, since    only differs from    by a multiplicative factor  1 , they always have the same sign.Thus, the model predicts the same preference no matters the choice of  and , meaning that the twostage problem and the single-stage problem (obtained by removing the common branch) are practically equivalent in the anticipated surprise model.

Appendix 3: Analysis of Blackjack gambling
The rule of Blackjack Here we are only describing the rules of the blackjack that are relevant to this work.Also please note that there are no official rules for blackjack as casinos are free to introduce their own house rules.However, the one we describe here is one of the best known, widely used in casinos and work in gambling analysis (Shackleford, 2019a).
Blackjack uses standard 52-card decks.In a casino setting where many games are played, multiple decks of cards are used and played cards are introduced back into the deck often.This is to minimize the change of the winning odds as a result of changes in the composition of the deck as cards are exhausted.The goal of the player is to obtain a hand of cards with a higher value than that of the dealer.The value of the hand depends on a point system.Points are calculated by summing up the numbers on all cards in the hand.Face cards (i.e.Js, Qs and Ks) stands for 10 points.For the Aces, they can stand for either 1 point or 11 points, chosen in order to maximize the value of the hand.The value of a hand in descending order is as follows: an Ace and a single card with 10 points (also known as blackjack (abbrev.BJ)), 21 points (but not BJ), 20 points, 19 points, 18 points, 17 points, 4-16 points, more than 21 points (known as busted hand) for dealer, busted hand for player.
In the beginning of a round, the player is given two cards and the dealer is given one card faced up and one card faced down (the player cannot read the card faced down).The player plays before the dealer, except when the faced-up card of the dealer is an Ace, in which case we will cover later.He can choose to take an extra card or not to.If he chooses to take an extra card, a card will be dealt to him, and the same option will be presented to him again until his hand become busted.If he chooses not to, the turn will be passed to the dealer.The dealer will keep taking extra cards until his hand has more than or equal to 17 points.After the dealer finished playing, the round ends, and whoever has a hand with a higher value wins.
When the player wins, he receives twice the amount of his original bet, thus giving him a net win of the size of his bet.An exception is when he wins with a hand of blackjack, in which case he will in addition get an extra amount of 0.5 times of his bet.When the player loses, he receives nothing, thus giving him a net loss of the size of his bet.In the event where the hand of the player and the dealer has equal value, normally the player will get back his bet and thus winning or losing nothing.However, to simplify our analysis, in section 3.1.1and 3.1.2.2, we assume that the player will instead throw a coin to decide if he wins or loses, giving him 50% chance of winning and 50% chance of losing an amount equal to the size of his bet.
If the face-up card of the dealer is an Ace, the player will be given an option to place a side bet of the size half his original bet, known as 'the insurance', to bet on whether the hand of the dealer is Blackjack.Before the player starts playing, the dealer will peek at the face-down card.If the dealer has blackjack, the player will get a net win of the amount 2 times of his side bet, which is equivalent to the size of his original bet.At this point, the original bet can also be resolved.If the dealer does not have blackjack, the player will lose his side bets and play resumes in order to resolve the original bet.The placement of the insurance side bet does not affect how the original bet is resolved.
Analysis for Section 3.11 (16 vs 10 situation) Here we are showing that Δ  ≥ Δ ℎ .First, we note that the size of  has no role in affecting the rank of ∆  and ∆   since we can make a transformation to remove  during the comparison between these two quantities.To simplify our analysis, we set  = 1 such that Δ  and Δ ℎ is given by: Since  is convex, f( 2 −  0 ) ≤ ( 2 ) − ( 0 ), eq.(A7) can then be rewritten as: For  = 1, the 1 st term in eq.(A8) vanishes.For the 2 nd term, define () = (1 − ) + (),  ∈ [0,1].
For  > 1, we note that ∆ is an increasing function with  since Therefore  ≥ 0 also holds for  > 1.
Analysis for Section 3.1.2.2 (taking side bets when the player has a nonblackjack good hand) Here we are showing that ∆  ≥ ∆   .We are assuming that both the options of taking and not taking side bets has the same expected value, this amounts to ). Denoting the initial expected value of the hand, i.e before the dealer peeks for BJ by  0 . 0 can be expressed as Like in the previous section, the size of  has no role in affecting the rank of ∆  and ∆   since we can make a transformation to remove  during the comparison between these two quantities.To simplify our analysis, we set  = 1 such that Let  1 be the expected value after the dealer peeks for BJ if the player takes the side bet.
Let  2 be the expected value after the dealer peeks for BJ if the player does not take the side bet.
It is trivial that  2 ≥  0 .However, whether it is  1 ≥  0 or  1 ≤  0 depends on the sign of  0 .Therefore, we divide the problem in the separate cases:  0 < 0 and  0 ≥ 0.
Case 1:  0 < 0 When  0 < 0,  1 <  0 .∆  and ∆   can be expressed as , we have For  = 1, using the properties of a convex function, we have It follows that  ≥ 0.
For  > 1, we again study the derivative of ∆ with respective to ) such that  increases with and remains positive at  > 1.
From eq. ( A11b), . Since a large  2 means that there is a large probability of winning with the hand, which, in order words, means the hand is a good hand.The results suggest that players prefer to take side bets when they have good hands.
Case 2:  0 ≥ 0 Since in the previous section, we established that  0 ≥ −1 2 is always considered good hands, we have to show that when  0 > 0, ∆ ≥ 0 unconditionally.Noting that when  0 ≥ 0,  1 ≥  0 , and that from eq. ( A11b)  0 has an upper bound of 1 3 , ∆  and ∆   is given by , we have For  = 1, again using the properties of a convex function, we have )) It follows that  ≥ 0.
From (A18), it is obvious that  increases with  such that  ≥ 0 still holds when  > 1.
Appendix 4: Analysis of the Ellsberg paradox Here we are exploring the condition in which the unambiguous urn would be preferred over the ambiguous urn.We assume that a symmetric prior probability that satisfies () = (1 − ) for  ∈ {0, and the ambiguous urn (∆ 2 ) is given by eqs.( 6a) and (6b) in the main text.For convenience, we are repeating them here: for general function .Based on these results, since  (and hence ) is a continuous function, (A21) would be true if for any convex function .Here we would explore possible conditions where  ′′ () < 0 is true.We note that We have plotted   against  for a variety of parameters ,  and , as shown in Figure S2.It can be observed that both the concavity of the value function  and probability weighting  raised  at small .The latter also suppressed  at large .Overall, this is again at odds with experimental results where  should be increasing with  and should not be positive at all .
As a comparison, the differences in anticipated surprise predicted by our model  (See eq.(A15)) using realistic parameters we suggested at Section 3.3 in the main text are shown in Figure S3.Consistent with our analysis at Appendix 3 and the experimental observations,  is increasing with  and is positive at large .

Figure S2 :
Figure S2: Differences in prospect between taking and not taking side bets (  ) at different  for different choices of parameters.Positive   implies preference for taking side bets while large  implies good hands.Top:  is fixed at 0.6 while the lines in the plots correspond to different values of .Bottom: Top:  is fixed at 0.6 while the lines in the plots correspond to different values of  .Left:  = 1.Right:  = 2.5.Note that no combination of parameters is consistent with the experimental observation that taking side bets is preferred when the player has a good hand.

Figure S3 :
Figure S3: Differences in anticipated surprise based on the model in this work between taking and not taking side bets () at different .Positive  implies preference for taking side bets.